It is frequently necessary to change the overall shape of a set of images comprising an image sequence. An example is modification of video material originated with an aspect-ratio (width to height ratio) of 16:9 to enable it be shown on a display device having an aspect ratio of 4:3. In such processing it is usually undesirable to alter the proportions of objects portrayed in the images. Known methods include removing (cropping) material from parts of the images, and applying spatial scaling which varies over the image area so that only objects near to the edge of the images are distorted.
More complex techniques include systems which analyse the image content so as to identify portrayed objects whose spatial proportions must be preserved, and only to scale other parts of the images to achieve the required change in the overall shape of the images.
A particularly interesting technique for re-sizing single images is described in the paper “Seam Carving for Content-Aware Image Resizing” by Avidan and Shamir, published in the Proceedings of the SIGGRAPH conference in August 2007. In this technique a path connecting opposite edges of an image via set of adjacent pixels having a minimum measure of cumulative high-spatial-frequency energy is identified as a “seam”. The pixels comprising this seam are then removed from the image, thus reducing the size of the image by one pixel-pitch in the direction orthogonal to the seam. This process is repeated on the processed image so as to identify a second seam, which is also removed; and the process continues until the desired size reduction has been obtained.
By using a measure of minimum high-spatial-frequency energy to identify seam positions, pixels are removed in relatively undetailed areas which are more likely to represent the spaces between portrayed objects, thus minimising the shape distortion of portrayed objects.
The paper also describes how different degrees of size reduction can be applied in the vertical and horizontal directions and how the technique can be extended to deal with expansion of the image size.
However, the application of these techniques to a sequence of related images, such as views of the same scene captured in a time or viewpoint sequence, leads to unpleasant artefacts.